A DUAL EIGENVECTOR CONDITION FOR STRONG 
LUMPABILITY OF MARKOV CHAINS* 
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Abstract. Necessary and sufficient conditions for identifying strong lumpability in Markov 
chains are presented. We show that the states in a lump necessarily correspond to identical elements 
in eigenvectors of the dual transition matrix. If there exist as many dual eigenvectors that respect 
the necessary condition as there are lumps in the aggregation, then the condition is also sufficient. 
The result is demonstrated with two simple examples. 
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1. Introduction. In this paper we present a method for coarse graming Markov 
chains by partitioning, or lumping, the state space into aggregated Markov chains. 
This approach to coarse grain Markov chains has been studied for a long time, see 
e.g. [71 [TT]. Due to the combinatorial explosion of the number of possible partitions 
of the state space, the conditions used to define lumpability are not useful for identi- 
fying aggregated states even for relatively moderately sized Markov chains. Existing 
efhcient methods for lumping are concerned with the special task of finding the most 
coarse approximate lumping [4] , or special situations where the transition matrix pos- 
sesses known symmetries (e.g. permutation symmetries) [2| [9| fTO]. There also exist 
efficient methods for testing that a given partition constitute a valid lumping [Ij [6l [5] . 
It should be noted that there can exist multiple different accepted lumpings of a 
Markov chain. Furthermore, different lumpings are in general not related through 
successive refinements. This paper addresses the general question: Given the transi- 
tion matrix of a Markov process, how can the state space be partitioned such that 
the Markov property is not lost in the aggregated process? 

2. Main result. Consider a Markov chain Xt, t — 0,1, ... , with a finite state 
space E = {1, . . . , N} and transition probability matrix P = [pij]. The transition 
matrix operates from the right Xt+i — xtP. A lumping is defined by a partition of 
the state space S — {Li, . . . , Lm} where Lk is a nonempty subset of S, n L; = 
a k I and IJj, Lk — S. By necessity M < N. A lumping can be defined by an 
N X M matrix 11 — [nik] where nik — 1 if i G Lk and nik — if i ^ Lk- The 
lumping S induces a quotient process Xt — xtH. If the process it is a Markov pro- 
cess (which is not usually the case), then we say that the Markov chain is strongly 
lumpahle with respect to 11 (or S). The following criterion is necessary and sufhcient 
for a Markov chain with transition matrix P to be lumpablc with respect to 11 or E [7] : 

THEOREM 1. //, for any Lk,Li G E, the total probability of a transition from any 
state i G Lk to Li, i.e. '^j^j^^Pij , is independent ofi, then P is lumpahle with respect 

to S. 
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The transition matrix for the reduced Markov chain is given by 

Pki = X! * ^ (^--^^ 

where we note that P = [pki] is well defined since the sum is independent of which 
representative i ^ Lk we chose. 

As mentioned in the introduction, Theorem 1 is not immediately useful for iden- 
tifying lumpings of a Markov chain. In previous work by Barr and Thomas [T], a 
necessary himpability criterion involving the left eigenvectors of the transition matrix 
was presented. It was first noted that the spectrum of P must be a subset of the spec- 
trum of P. Furthermore, it was noted that if vP = Xv then vP = Xv, with v = vll. 
It follows that if A is an eigenvalue of both P and P, then v is an eigenvector of P, 
but if A is not an eigenvalue of P then v = vH — 0. Intuitively this observation can 
be understood as 11 eliminating a subset of the eigenvectors of P. It is an example of 
a general result concerning reduction of a linear operator P, which can be formulated 
as a commutation relation HP — PH that must be fulfilled for a reduction 11 to be 
well defined 0. It is implied that ker(n) is P- invariant (i.e. spanned by eigenvectors 
of P, if we for the moment ignore the problem of degenerated eigenvalues). 

Barr and Thomas' result suggests a search for lumpings defined by 11 such that 
w^n — for some subset of the left eigenvectors of P, {w^jagj, J C E. Since 11 
should be a matrix with zeros and ones, v^H = essentially means searching for 
eigenvectors with subsets of elements that sums to zero. For lumpings only involving 
agglomeration of two states this is straightforward since the eigenvector (s) must have 
pairs of elements vf — — w". However, agglomeration of k states means searching for 
partial sums evaluating to zero and involving k elements. In addition (as we will see) 
there must be several eigenvectors simultaneously satisfying v^Il — (the criterion 
is only necessary, not sufficient). As Barr and Thomas point out, there is no obvious 
algorithm to generate 11 based on their result. 

In this paper we present a result that can easily be used for identifying possi- 
ble partitions of the state space leading to a reduced Markov process. Our method 
is based on the observation that the dual of the probability space can be used to 
identify lumplings. The criterion w^II = is viewed as an orthogonality condition 
between the left eigenvectors j and the column space of 11. The orthogonal 

complement of a set of left eigenvectors is spanned by complementary right eigenvec- 
tors (defined naturally in the dual vector space). These complementary eigenvectors 
span the column space of 11. Requiring that 11 consists of zeros and ones does, as we 
will see, correspond to a criterion of repeated elements within each complementary 
right eigenvector. Clearly, identifying repeated elements in the right eigenvectors is 
algorithmically straight forward. The precise result reads as follows. 

THEOREM 2. Assume that P is a diagonalizable transition matrix with full ran^ 
describing a Markov process Xt+i — XtP ■ Consider a set of linearly independent right 
eigenvectors of P, Pu" — A"it". Let I C Y, be the set of indices for the eigenvec- 
tors. Form state equivalence classes defined by states with identical elements in all 
eigenvectors u", i.e. i ^ j iff uf — Va G /. The equivalence classes define a 

partitioning E of the state space. This partitioning is a lumping of the Markov chain 
if the number of partition elements equals the number of eigenvectors, i.e. \T,\ ~ \I\. 

^The transition matrix (1 — f)P + fl, < f < 1, allows exactly the same lumping as P according 
to Theorem 1. The rank condition is therefore not a real restriction. 
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Conversely, ifY, is a lumping then there exist |S| linearly independent right eigen- 
vectors that are invariant under permutations within the lumps. 

Proof. Let {w''}/3=i,...,Ar be the linearly independent (1 x N) left eigenvectors of P and 
{u^}(3=i....,N be the {N x 1) right eigenvectors. We normalize the left and/or right 
eigenvectors so that v^u'^ = 5^'^ (where 6 is the Kronecker delta). Let H define the 
partitioning E — . . . , ^|/|}- Since are linearly independent and jSj = |/|, 

by the construction of the equivalence classes, the column space of H is also spanned 
by {""lag/. From the orthogonality of left and right eigenvectors it follows that 

w'^n = 0, V/3 ^ /. (2.2) 

We now decompose P as 

N 

K,=^A^ufz;f, (2.3) 

0=1 

where uf and iif denote the ith entry in right respective left eigenvector /3. Let i £ Lk- 
From Eg. [Ql follows that 

N N 

Y: p.: - E E «f = E ^'-^ = E ^'-^ ' (2-4) 

jeLi 13=1 jeLi 13=1 i3ei 

where we use Eg. 12.21 in the last step. By the construction {u^ — Uj if i, j Lk, 
P € I) it is clear that the right hand side of Eg. l2.4l does not depend on i e Lk. From 
Theorem 1 it follows that E is a lumping of the Markov chain. 

We now show that if the Markov chain is lumpable with E defining the partition, 
then there exist |E| eigenvectors with constant elements over the lumps. Let E = 
{Li, . . . , be a lumping of the Markov chain. Let y be a vector defined as 

Vi=yk i ^ Lk, (2.5) 

i.e. constant elements within the lumps defined by E, and y at this point an arbitrary 
vector on the reduced state space. Now consider 

E^^^i^J = E E = E^'^'?/; i&Lk. (2.6) 

j I jeLi I 

It follows that y is a right eigenvector of P if y is an eigenvector of P. Since P has 
exactly jEj eigenvectors (clearly, P is diagonalizable if P is) it follows that P has |E| 
eigenvectors with the structure ^mposed in Eg. 12.51 □ 

The two trivial lumpings: E = {Li, . . . ,Lff} (all states in different lumps) rep- 
resented by all the right eigenvectors (since P is full rank there can be no index pair 
equal in all eigenvectors); and E = {Li} (all states in one lump) represented by the 
always present u = (1,1,...,!)"^ eigenvector (implied by '^jPij = !)■ As a conse- 
quence of E = {Li} always being an accepted lumping, any eigenvector suggesting a 
unique two-lump partition is an accepted lumping. In practice, the right eigenvectors 
involved in forming the equivalence classes often, but not always, form a hierarchy of 
refined partitions (always including E = {Li} as the most coarse partition), and the 
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overall partition is set by the eigenvector suggesting the finest partition. It should 
be noted however that nothing prevents = ui for k ^ I in Eq. 12.51 which implies 
that not all lumpings of a Markov chain must correspond to a single eigenvector with 
exactly that suggested lumping. A lumping can instead be composed of eigenvectors 
each by themselves suggesting a coarser lumping. Consider for instance (1,1,..., 1)^, 
which is always a member of a subset of eigenvectors that correspond to a valid lump- 
ing. The four-lump partition (S5) in Example 2 below also exemplifies this situation. 
In general, the total number of possible lumpings of a Markov chain may exceed the 
number of eigenvectors, i.e. the number of states. 

Theorem 2 is valid also if P has redundant eigenvalues. However, eigenvectors cor- 
responding to redundant eigenvalues are not unique. By "rotating" the eigenvectors 
corresponding to a redundant eigenvalue it can be possible to find different suggested 
partitions. Especially, any k elements can be set equal by choosing an appropriate 
linear combination of the k eigenvectors corresponding to a fc-fold degenerate eigen- 
value. It can happen that forcing a subset of the elements to be equal simultaneously 
sets some other elements equal as well, creating a more coarse lumping than expected. 
The degenerate eigenvalues in Example 2 demonstrates this point. 

The identity matrix is an extreme example of redundant eigenvalues. In this case 
every partition of the state space is a lumping. The number of possible lumpings 
equals the Bell number Bn and grows extremely fast with N. From this argument it 
is clear that, in some cases, generating all possible lumpings of a Markov chain is not 
computationally feasible. The lumpings suggested by the eigenvectors can be viewed 
as " generators" that can possibly be combined to create finer partitions of the state 
space. A more non-trivial example than the identity matrix is given in Example 2. 

3. Examples. Theorem 2 is illustrated with two examples. 



Example 1. Consider the transition matrix 



/ a + 6 + (c - l)/2 1-a-b (1 - c)/2 \ 
P= -a+(c+l)/2 a (l-c)/2 , 

\ 1-6-c b c / 

with < a, 6, c < 1. P has the eigenvalues = 1, = 2a-f 6- 1 and A^ — (3c- l)/2, 
and the eigenvectors 

= (1,1, if, 

= (l + c-2a-26, 2(a-c),26 + c-l)'^ and 
z.^ = (-l,-l,2r. 

There are three possible lumpings of P: 

51 - {{1,2,3}} from{ui}, 

52 = {{1,2}, {3}} from {ui, 1*3} and 

53 = {{1},{2},{3}} bom {u\u\u^}, 

where S2 is valid if 2a + — 1 ^ and 3c — 1 7^ 0. 



Example 2. Consider the transition matrix (from '3^) 
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Table 3.1 

Possible lumpings of P and associated subsets of eigenvectors. 



Lumping 


Subset of eigenvectors 


51 = {{1,2,3,4,5,6,7,8}} 

52 = {{1,2,3,4},{5,6,7,8}} 

53 = {{1,2},{3,4},{5,6,7,8}} 

54 = {{1,2,3, 4},{5,6},{7,8}} 

55 = {{1,2},{3,4},{5,6},{7,8}} 

56 = {{1,2},{3},{4},{5,8},{6,7}} 
I]7 = {{1},{2},{3,4},{5,7},{6,8}} 
S8 = {{1,3},{2,4},{5,6},{7},{8}} 
I]9 = {{1,4},{2,3},{5},{6},{7,8}} 
Eio = {{l},{2},...,{7},{8}} 


{«!} 

{u-^, u^, u'', u®} 
{w^, u^, u^, u'', u^} 

{^i^u^^i^u^uS} 
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where a = 1/5 and b = 1/4. P has two pairs of degenerate eigenvalues. The associated 
(not normahzed) right eigenvectors of P have the following structure (columns 
sorted in order of decreasing |A|): 
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-5/4 ) 



for some numerical constants ci and C2, (ci sa 0.579 and C2 ~ 1.079). Note that 
and it^ share eigenvalue and therefore any linear combination au^ + jSu^ is also an 
eigenvector. The same is true for and . 

The possible lumpings of P and the corresponding subsets of right eigenvectors 
are given in Table [XT] Note that S5 is a lumping with an agglomeration not suggested 
by any of the individual eigenvectors involved, but rather by the set of eigenvectors 
as a whole. Further, partition Sg through Sg involve appropriate linear combinations 
of eigenvalues that correspond to the two degenerate eigenvalues of P, as indicated in 
Table O 

4. Discussion and conclusions. We have presented a sufficient and necessary 
condition for Markov chain lumpability. The condition can be applied to identify 
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a set of lumpings of a given Markov chain by identifying sets of identical elements 
in the dual eigenvectors. The results presented could have been expressed naturally 
using symmetries. The equivalence relation between identical elements in the dual 
eigenvectors implies invariance under a permutation symmetry. The lumpings are 
orbits of the permutation group, which acts as a normal subgroup extension to the 
transition matrix. In some situations this formulation can be advantageous, especially 
to clarify correspondence with general reduction methods in dynamical systems. In 
this paper we have chosen not to emphasize the symmetry formulation since it tends 
to obstruct the clarity of the presentation. 
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